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Abstract 

We describe an algorithm that computes the index of a finitely gen- 
erated subgroup in a finitely L-presented group provided that this index 
is finite. This algorithm shows that the subgroup membership problem 
for finite index subgroups in a finitely L-prescnted group is decidable. As 
an application, we consider the low-index subgroups of some self-similar 
groups including the Grigorchuk group, the twisted twin of the Grigorchuk 
group, the Grigorchuk super-group and the Hanoi 3-group. 

Keywords: Coset enumeration; recursive presentations; self-similar groups; 
Grigorchuk group; low-index subgroups. 

1 Introduction 

Many algorithmic problems are unsolvable for finitely presented groups in gen- 
eral. For instance, there is no algorithm which allows to decide if a group given 
by a finite presentation is trivial [21] . However, the coset enumeration process 
introduced by Todd & Coxeter [27] and investigated by various others, see [19] 
or the historical notes in Chapter 5.9 of [26J, computes the index of a finitely 
generated subgroup in a finitely presented group provided that this index is 
finite. Therefore, the Todd-Coxeter method allows one to prove that a finitely 
presented group is trivial. 

Coset enumeration is one of the most important tools for investigating 
finitely presented groups; but, if the subgroup has infinite index, this pro- 
cess will not terminate. Even if the subgroup has finite index, there is no 
upper bound on the complexity of coset enumeration. Therefore, even prov- 
ing a finitely presented group being trivial is computationally a challenging 
problem [191125]. 

For this reason, solving algorithmic problems for infinitely presented groups 
seems entirely infeasible. However, an interesting family of recursively presented 
groups was recently shown to be applicable for computer investigations. Exam- 
ples of such groups arise as subgroups of the automorphism group of a regular 
tree. A famous example is the Grigorchuk group <3 which plays a prominent 
role in the area of Burnside problems [IB]. The group <3 is finitely generated 
and it admits a recursive presentation whose relations are given recursively 
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by the action of a finitely generated free monoid of endomorphisms acting on 
finitely many relations [22]. Infinite presentations of this type are called finite 
L -presentations in honor of Lysenok's latter result for the Grigorchuk group (5; 
see Section UJ or [T] for a definition. 

Finite L-presentations are 'natural' generalizations of finite presentations 
and, as the concept is quite general, they found their application in various 
aspects of group theory; see, for instance, [H1[2DJ[23]. A finite L-presentation of 
a group allows to compute its lower central series quotients [3] and the Dwyer 
quotients of its Schur multiplier |18j . The Dwyer quotients often exhibit period- 
icities which yield detailed information on the structure of the Schur multiplier 
in general. 

In this paper, we describe a coset enumeration process for computing the 
index of a finitely generated subgroup in a finitely L-presented group provided 
that this index is finite. In order to achieve this method, we show in Section [3] 
that finitely many relations are sufficient to compute an upper bound on the 
index using coset enumeration for finitely presented groups. It then remains to 
either prove that this upper bound is sharp or to improve the bound otherwise. 
In Section U we show that the latter problem is algorithmically decidable in 
general. In particular, we show that there exists an algorithm which decides 
whether or not a map from the free group over the L-presentations generators 
into a finite group induces a homomorphism from the L-presented group. 

Similar to coset enumeration for finitely presented groups, our method 
for finitely -L-presented groups allows straightforward applications including 
a membership test for finite index subgroups. In particular, our method allows 
us to compute the number of subgroups with small index for some self-similar 
groups in Section [5j Our explicit computations correct the counts obtained 
in OH], and hence we provide a further step towards Problem 6.1 raised in |14j . 

We have implemented our coset enumeration method and its applications 
in the computer algebra system Gap |13j . Computer experiments with this 
implementation demonstrate that our method works reasonably well in practice. 

In a forthcoming paper, we prove a variant of the variant of the Reidemeister- 
Schreier theorem for finitely L-presented groups which shows that each finite 
index subgroup of a finitely L-presented group is finitely L-presented itself. 

2 Preliminaries 

We briefly recall the notion of a finite L-presentation as introduced in [I]. For 
this purpose, let F be a finitely generated free group over the alphabet X. 
Furthermore, let Q, 1Z C F and $ C End(i ? ) be finite subsets. Then the 
quadruple (X \ Q | $ | 1Z) is a finite L-presentation. It defines the finitely 
L-presented group 



\ <tG$* / 

where <3?* denotes the free monoid of endomorphisms generated by that is, 
the closure of {id}U$ under composition of endomorphisms. We will also write 




(1) 
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G = (X | Q | $ | TZ) for the finitely L-presented group in Eq. (pQ). 

Clearly, every finitely presented group (X \ TZ) is finitely L-presented by 
{X | | | TZ). Therefore, finite L-presentations generalize the concept of 
finite presentations. Other examples of finitely L-presented groups are various 
self-similar groups or branch groups [1]. For instance, the Grigorchuk group 
satisfies the following 

Theorem 1 (Lysenok, 1985) The Grigorchuk group (3 is finitely L-presented 
by 

({a,b,c,d} | {a 2 , b 2 , c 2 , d 2 , bed} \ {a} \ {(ad) 4 , (adacac) 4 }) , 

where a is the endomorphism of the free group over the alphabet {a, b, c, d} 
induced by the mapping a *— > aca, b i— > d, c^rb, and d i— > c. 

Proof. For a proof, we refer to [22]. □ 

Finite L-presentations are recursive presentations which are 'natural' general- 
izations of finite presentations. They were used by various authors to construct 
groups with interesting properties; see, for instance, [§1 \'2U\ 125]. Furthermore, 
every free group in a variety of groups that satisfies finitely many identities is 
finitely L-presented pQ; e.g., the free Burnside group B(n,m) of exponent m 
on n generators is finitely L-presented by 

({ ai ,...,a n }U{t}\{t}\Z\{t m }), 

where the endomorphisms £ = {a x \ x E {of 1 , • • • , a^ 1 }} are induced by the 
mappings 

J at i — ^ ai, for each 1 < i < n 
x ' \ t i — ¥ tx, 

for each x € {af l , . . . , a^ 1 }. 



3 Coset enumeration for finitely L-presented groups 

Let G = (X | Q | $ | TZ) be a finitely L-presented group and let U < G be 
a finitely generated subgroup with finite index in G. In this section, we show 
that coset enumeration for finitely presented groups yields an upper bound on 
the index [G :U\. In Section HI it then remains to prove (or disprove) that this 
upper bound is sharp. 

Let {gi, . . . ,g n } be a generating set for the subgroup U. We assume that 
the generators of U are given as words over the alphabet X U X~ . Denote the 
free group over X by F and let K be the normal subgroup 

K=/qu [J TZ' 

so that G = F/K holds. Then the subgroup E = (gi, ... , g n ) < F satisfies that 
U = EK/K. Hence, we are to compute the index [G : U] = [F : EK]. 
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For an element a € <&*, we denote by ||a|| the usual word-length in the 
generating set <3? of the free monoid Define <J>W = {a 6 f * ||c|| < i}, for 
each i £ Nq. Then, as Q, and 7£ are finite sets, the normal subgroup 



is finitely generated as normal subgroup. We obtain K = [J i>0 Ki and also 
EK = [J i>0 EKi. Consider the ascending chain of subgroups 



Then the following lemma is straightforward. 

Lemma 2 The subgroup EK has finite index in F if and only if there exists 
£ € N such that EKe has finite index in F. In that case, there exists £' € N such 
that EK V = EK. 

Proof. Obviously, if [F : EKg] is finite for some £ £ N, then the subgroup EK 
has finite index in F. On the other hand, if [F : EK] is finite, then, as F is 
finitely generated, the subgroup EK is finitely generated. Let {u±, . . . ,u n } be 
a generating set of EK. Since EK = \J i>0 EKi holds, there exists a positive 
integer £ £ N such that {u\, . . . , u n } C EKe and thus EKn = EK. □ 

Note that the index [F : EKj>] is the index of the subgroup hi in the finitely 
presented covering group 



By Lemma [2j there exists a positive integer £ £ N so that the subgroup hi 
has finite index in G%. In this case, coset enumeration for finitely presented 
groups computes the index [Ge : hi]. Although we do not know this integer £ 
a priori, we can use the following firsthand approach to find such an integer: 
Starting with £ = 1, we attempt to prove finiteness of [Ge : hi] using coset 
enumeration for finitely presented groups. If this attempt does not succeed 
within a previously defined time limit, we increase the integer £ and the time 
limit. We continue this process until eventually the index [Ge : hi] is proved 
to be finite. In theory, Lemma [2] guarantees that this process will terminate. 
Computer experiments with the implementation of our method in Gap show 
that this firsthand approach works reasonably well in practice. In particular, 
our implementation allows to compute the index of all subgroups considered 
in [3 HE] and Chapter VIII of [TO]. 

Suppose that the integer £ € N is chosen so that n = [Ge : hi] is finite and 
that the coset enumeration for finitely presented groups has terminated and 
has computed a permutation representation ipf. F — >■ S n for the group's action 
on the right cosets EK e \F. Then the index [G : U] = [F : EK] divides the 
index [Ge ■ hi] = [F : EKe], and hence [Ge : hi] is an upper bound on [G : hi]. 
It therefore remains to either prove that [F : EK] = [F : EKe] holds, or to 




EKq < EK\ < ...< EK e < EK e+ i < . . . < EK < F. 



Ge = (X\{q,r CT \qeQ,r£K,o-£ $W}>. 



(2) 
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increase the integer I otherwise. The permutation representation tpf.F — > S n 
is called valid, if [F : EK] = [F : EK e ] holds. 

Clearly, a permutation representation ipf.F — > S n is valid if and only if 
every relation r £ F of the group presentation is contained in the kernel of <pi. 
Therefore, if the group G = F/K were finitely presented, only finitely many 
relations need to be considered to prove validity of (p£. However, for finitely 
L-presented groups, even checking validity of a permutation representation ipg 
involves possibly infinitely many relations. In Section [3J we prove that the latter 
problem is decidable in general. 

4 Deciding validity of a permutation representation 

In this section, we describe our algorithm for deciding whether or not a per- 
mutation representation tp: F —> S n , as considered in Section [3l is valid. This 
is equivalent to checking whether a coset-table for U in Gg obtained by the 
methods of Section defines the given subgroup U < G. 

Let ip: F — > S n be a permutation representation as in Section [3] and let <3?* be 
the free monoid generated by a finite set $ C End(i ? ). For two endomorphisms 
a G 3>* and 5 € <&*, we say that 5 reduces to a with respect to <p if there 
exists a homomorphism it: im(<7<^) — > im(5ip) such that aipir = Sep. In this case, 
we will write 5 a. Note that ~» is a reflexive and transitive relation on 
the endomorphisms <&*. The following lemma gives an equivalent definition for 

5 <7. 

Lemma 3 Let 5, a € End(-F) 6e given. Then 5 reduces to a with respect to ip 
if and only ifker(o~ip) < ker(5ip) holds. 

Proof. Assume that 5 °~ holds. Then, by definition, there exists a homo- 
morphism 7r: im(o~(p) — > im(S(p) such that aipir = Sip. Let g € ker(cr<^). Then we 
have that g Sv = g aip7T = (g^^Y = 1 and hence, we obtain g € ker(5</?). Suppose 
that ker(<7(/?) < ker(^) holds. Then we have the isomorphisms -F/ker(<79?) — >• 
im(<7</?), gkei(aip) i— > g "^ and -F/ker(<5(^) — > im(5y>), <7ker(<fy?) 1-4 g* 5 ^. We fur- 
ther have the natural homomorphism F/ ker (o~tp) — > F/ker(5<^), gker(o~ip) 1— >• 
gker(<5i^). This yields the existence of a homomorphism ir:im(o~<p) — > im(<fy?) 
such that s^ 71 " = 5^. □ 

A finite generating set for the kernel ker(cr</?) is given by the Schreier theo- 
rem |2H Proposition 3.7] and hence, it is straightforward to check whether or 
not 5 o~ holds. The definition <5 a also yields the following immediate 
consequence. 

Lemma 4 There is no infinite set of endomorphisms of F such that for each 
pair (a, 5) from this set, neither a 5 nor 5 a hold. 

Proof. Obviously, for every endomorphism a £ End(i ? ), it holds that a a. 
By the universal property of the free group F, a homomorphism o~<p: F — > S n is 
uniquely defined by the images , . . . , x^f of the elements x\, . . . , x n of a basis 
of F. Since im(c^) is a finite group, there are only finitely many homomorphisms 
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F — > im(yj) and therefore Hom(.F, im(^)) is finite. Hence, an infinite set of 
endomorphisms will contain endomorphisms a and 5 with = , for each 
1 < i < n. In this case, a 5 obviously holds. □ 

An element a € <I>* is called a ^-descendant of 5 € 3>* if there exists -0 € $ 
such that a = ip5. Thereby, the free monoid $>* obtains the structure of a 
| $ [-regular rooted tree with its root being the identity map id: F — > F. We can 
further endow the monoid <£* with a length-plus- (from the right)-lexicographic 
ordering -< by choosing an arbitrary ordering on the finite set More precisely, 
we define a ~< 5 if ||cr|| < \\S\\ or, otherwise, if a = o\ • • • a n and 5 = S± ■ ■ ■ S n , 
with Gi, Si € and there exists a positive integer 1 < k < n such that o% = Si, 
for k < i < n, and -< <5fc. Since <3? is finite, the obtained ordering -< is 
a well-ordering on the monoid see [26] . and therefore there is no infinite 
-^-descending series of endomorphisms in 

Our algorithm that decides validity of a permutation representation ip: F — s> 
S n is displayed in Algorithm [1] below. We need to prove the following 

ISVALIDPERMREP^, Q, 71, W, <p) 
Initialize V := {id: F ->• F} and S 1 := $. 
Choose an ordering on $ = {fa, . . . , re } with fa ~< fa+\. 
while S ^ do 

Remove the first entry 5 from S 1 . 
if (3r € 72.: r s ^ ker<£>) then return( false ) 
if not (3 a €V: 5 ^ v a) then 
Append fad, . . . , 4> n 5 to 5. 
Add 5 to V. 
return( true ) 

Algorithm 1: Deciding validity of a permutation representation 

Theorem 5 The algorithm IsValidPermRep returns true if and only if the 
permutation representation ip: F — > S n is valid. 

Proof. The ordering -< on <]? can be extended to an ordering on <I>* as described 
above. By construction, the stack S is ordered with respect to -<. Since F is 
finitely generated, the set of homomorphisms Hom(i ? , S n ) is finite. Thus, in 
particular, the set {dip \ 5 € V} C Hom(F, S n ) is finite and therefore the algo- 
rithm IsValidPermRep can add only finitely many endomorphisms to the set 
V. Thus, for every ^-descendant 5 in the stack S, there will eventually exist an 
element a €V such that 5 a. Therefore, the algorithm IsValidPermRep 
is guaranteed to terminate and it returns either true or false. Clearly, if the 
algorithm returned false, then it found a relation r* 5 which yields a coincidence, 
and hence the permutation representation ip: F — > S n is not valid. 

Suppose that the algorithm returned true. By the constructions of Sec- 
tion [3l the fixed relations in Q of the L-presentation {X \ Q | $ | 1Z) are 
already contained in the kernel of the permutation representation ip. There- 
fore, it suffices to prove that every relation of the form r ai , with r £ 1Z and 
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ci G <&*, is contained in the kernel of (p. By construction, there exists 5 G V 
maximal subject to the existence of w G $* such that o\ = w5. If \\w\\ = 0, 
then o\ = 5 is contained in V and therefore r s G ker ip, as the algorithm did not 
return false. Otherwise, there exist tp G $ and w G 3>* such that w5 = vip5. 
Since ip5 G" V, there exists an element e £ V with e -< ip5, by construction, 
such that tpS reduces to e with respect to (p. Thus, by definition, there exists a 
homomorphism tt: im(ep) — > \m{ip5p) such that ip5p = epir. In particular, we 
obtain that r ailp = r w5ip = r v ^ Slp = r V£,pn . As tt is a homomorphism, it suffices 
to prove that r V£ G ker p. Note that, since e -< we have that v e -< = <ti. 
Continuing this rewriting process with the element CJ2 = ve yields a descending 
sequence o\ >- o"2 >- . . . in the monoid As the ordering -< is a well-ordering, 
this process terminates with an element a n G V . Since the algorithm did not 
return false, we have that r°" n G ker</? which proves the assertion. □ 

Note that, if the algorithm IsValidPermRep found a coincidence, this can be 
used to update the coset-table, and thus another application of coset enumer- 
ation for finitely presented groups can be avoided. Moreover, the Algorithm Q] 
yields the following 

Theorem 6 Let G be finitely L-presented by (X \ Q \ $ \ 1Z) and denote the 
free group over X by F . There exists an algorithm which decides whether or 
not a homomorphisms p:F^- S n induces a homomorphism G — > S n . 

If = {cr}* is generated by a single element a G End(i ? ), then there will exist 
positive integers < % < j such that o~ 3 a 1 . In this case, the algorithm 
IsValidPermRep simplifies to the following 

Corollary 7 Let < i < j be positive integers such that a 3 ~~*u> a 1 . Then we 
have [F : EK e ] = [F : EK] if and only if 

{q, r ak | q G Q, r G K, < k < j} C ker <p. (3) 
We consider the following 

Example 8 Let G denote the Basilica Group \1T$ . Then G is finitely L- 
presented by ({a,b} \ | {cr} | {[a, a 6 ]}), where a is induced by the mapping 
a i— > b 2 and b i— > a; see JSj/- We consider the subgroup U = (a 3 , 6, aba). A coset 
enumeration for finitely presented groups yields that the subgroup hi has index 
3 in the finitely presented covering group 

G = ({a,b}\{[a,a b ]}). 

Furthermore, we obtain the permutation representation ip: F — > £3 for the 
group's action on the cosets EKq\F. This permutation representation is in- 
duced by the mapping 

a (1,2,3) and 6^(2,3). 
We now obtain the images 

a** = (), b°* = (1,2,3), 



= (1,3,2), b°* = (), 
a° Z ^ = (), = (1,3,2) 
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Clearly, the mapping a a ^ h- > a a f and b a ^ h- > 6 CT ^ induces a homomorphism 
it: im(aip) — > im(<7 3 </?) ; and hence we have cr 3 <r. Ly Corollary^ it therefore 
suffices to prove that 

([a,a b ]r = (), ([a,a fe ]r = (), and ([a,a b ]f* = () 

/ioW. XTiis yields that [G : U] = 3. 

5 Further applications 

The permutation representation (p: F — > 5 n for a finite index subgroup EK/K < 
F/K yields various algorithmic applications. For instance, an element w £ F is 
contained in the given subgroup EK if and only if it stabilizes the trivial coset 
EK1. This can be easily be checked using the permutation representation <p. 
In particular, we obtain 

Theorem 9 The subgroup membership problem for finite index subgroups in a 
finitely L-presented group is decidable. 

Moreover, having computed permutation representations </?i and ip2 for two 
finite index subgroups U and V of a finitely L-presented group, one can compute 
a generating set for the intersection U D V. Thus, in particular, our method 
allows one to compute the core of a finite index subgroup. For example, the 
core of the subgroup U in Example [8] is given by 

H = (6 2 , a 3 , a 2 ba- 1 b~ 1 , abab~ 1 ,ab 2 a- 1 ,ba 2 b~ 1 a- 1 ,baba~ 2 }. 

Since H has finite index in G, our method allows to compute a permutation 
representation for the core H and we obtain G/H = £3. 

5.1 Low-index subgroups of finitely L-presented groups 

The coset enumeration process for finitely presented groups was used in 
to describe a low-index subgroup algorithm that computes all subgroups of 
a finitely presented group up to a given index. This algorithm also yields a 
method for computing all subgroups with small index in a finitely L-presented 
group. In this section, we will describe this method for finitely L-presented 
groups and we use this algorithm to investigate some self-similar groups. In 
particular, our implementation in the computer algebra system Gap allows us 
to determine the number of subgroups with index at most 64 in the Grigorchuk 
group. 

Let G = F/K be a finitely L-presented group and let n be a positive integer. 
Using the low- index subgroup algorithm for finitely presented groups jllj . see 
also Chapter 5.6 of [26], we obtain the list of subgroups with index at most n in 
the finitely presented covering group Gt = F/Kg. Since the covering group Gg 
naturally maps onto G, every subgroup EKg/Kg with index n in Gi maps to a 
subgroup of the finitely L-presented group G. The index of this image EK/K in 
G divides the index n = [F : EK(\. On the other hand, every subgroup EK/K 
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with index n in the finitely L-presented group F/K has a full preimage EKjKg 
in the finitely presented group Gi with index n. Thus the list of subgroups with 
index at most n in a finitely L-presented group G can be obtained from the list 
of subgroups of a finitely presented covering group Gi by removing duplicate 
images. Our solution to the subgroup membership problem can be used to 
remove duplicate images in G. 

As an application, we consider some interesting self-similar groups and we 
determine the number of subgroups with small index. We first consider the 
Grigorchuk group ©: its lattice of normal subgroups is well- understood [2] 
while its lattice of subgroups with finite index is widely unknown p3] • It is well 
known [T3| that the Grigorchuk group has seven subgroups of index two. In |25j . 
it was shown that the subgroups of index two are the only maximal subgroups 
of ©. Our low- index subgroup algorithm allows us to determine the number 
of subgroups with index at most 64 in the group © and thereby, it corrects 
the counts in Section 7.4 of [5] and in Section 4.1 of [J]. The following list 
summarizes the number of subgroups (<) and the number of normal subgroups 
(<) among them: 



index 


1 


2 


4 


8 


16 


32 


64 


< 


1 


7 


31 


183 


1827 


22931 


378403 


< 


1 


7 


7 


7 


5 


3 


3 



The Grigorchuk super-group © was introduced in [3] . It contains the Grigorchuk 
group as an infinite index subgroup. Little is known about its subgroup lattice. 
The twisted twin © of the Grigorchuk group was introduced in [B]. Similarly, 
little is known about the subgroup lattice of the twisted twin ©. Our low-index 
subgroup algorithm allows us to determine the number of subgroups with index 
at most 16 in both groups. Their subgroup counts are: 



index 


© 

< 


< 


© 

< 


< 


1 


1 


1 


1 


1 


2 


15 


15 


15 


15 


4 


147 


35 


147 


35 


8 


2163 


43 


1963 


43 


16 


52403 


55 


46723 


47 



As both groups are 2- groups, the only maximal subgroups with finite index are 
the subgroups with index two; though the question of dertermining all maximal 
subgroups of © and © has not been addressed in this paper. 

Finally, we consider the Basilica group and the Hanoi-3 group [15] with its 
L-presentation from [7j. The following list also includes the number of maximal 
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subgroups (max): 



. . Hanoi— 3 Basilica 

index 

< < max < <l max 



1 


1 


1 


1 


1 


1 


1 


2 


7 


7 


7 


3 


3 


3 


3 


12 


o 


12 


7 


4 


7 


4 


59 


7 


4 


19 


7 


o 


5 


15 


o 


15 


11 


6 


11 


6 


136 


4 


o 


39 


13 


o 


7 


21 


o 


21 


15 


8 


15 


8 


335 


13 


o 


163 


19 


o 


9 


225 


o 


o 


115 


13 


9 


10 


153 


3 


o 


83 


19 


o 


11 


33 


o 


33 


23 


12 


23 


12 


2872 


12 


o 


355 


31 


o 


13 


39 





39 


27 


14 


27 


14 


297 


3 


o 


115 


25 


o 


15 


450 


o 


o 


77 


24 


o 


16 


1855 


13 


o 


1843 


47 


o 


17 


51 


o 


51 


35 


18 


35 


18 


5001 


3 


o 


1047 


44 


o 


19 


57 


o 


57 


39 


20 


39 


20 


1189 


9 





939 


45 





21 


756 








105 


32 





22 


531 


3 





223 


37 





23 


69 





69 


47 


24 


47 


24 


52220 


23 





4723 


87 





25 


225 





75 


411 


31 


25 


26 


783 


3 





315 


43 





27 


5616 





27 


736 


49 





28 


2301 


9 











29 


87 





87 








30 


15462 


3 











31 


93 





93 








32 


9119 


25 












The largest abelian quotient H/H' of the Hanoi-3 group H is 2-elementary 
abelian of rank 3. Thus, by the Feit-Thompson theorem |12| . there are no 
normal subgroups with odd index in the Hanoi-3 group. 
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